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Executive Summary 

With the release of Ohio's state test score data each August, one recurring question is how well have the state's 
large sector of charter schools performed relative to their counterparts in traditional districts? The Thomas B. 
Fordham Institute commissioned Public Impact to conduct an analysis of the 2009-10 data in this report. 

Using public academic performance data from the Ohio Department of Education, the analysts compared the 
performance of urban public charter schools with that of non-charter public district schools in the state's eight 
major urban cities, the "Ohio 8" (Akron, Canton, Cincinnati, Cleveland, Columbus, Dayton, Toledo, and 
Youngstown), where most brick-and-mortar charter schools reside. Separately, they compared the performance of 
charter e-schools (also called virtual schools) with that of non-charter public schools statewide. The analysis 
examined both "growth" (based on how much students learned over the school year) and "achievement" (based 
on the percentage of students meeting state standards). 

Among the key takeaways: 

• Only two percent of charters and two percent of district schools had high growth and high achievement - 
an even lower percentage than last year. While urban charters tended to show stronger growth than their 
district counterparts on average, neither sector showed particularly strong achievement, with only seven 
percent of charters and four percent of district schools in the highest tier of the Performance Index. 

• Among schools with low achievement and low growth in 2007-08, charters were far more likely to 
improve by 2009-10, with 23 percent showing enough improvement to make moderate achievement and 
above expected growth, compared with only two percent of district schools. Still, most of 2007-08's 
struggling schools remained low-performing in 2009-10. Seventy-one percent of these charter schools and 
90 percent of these district schools remained within the bottom Performance Index tier. 

• Urban charter schools had a slightly higher percentage of students scoring proficient on state tests in 
reading and math (for the first time since 2005-06). But too few students, charter or district, were 
proficient in either subject. In reading, 62 percent of charter students were proficient compared with 59 
percent of district students. In math, only 50 percent of charter students were proficient, compared with 
49 percent of district students. 

• In both Cleveland and Dayton, charters outperformed their district counterparts. In Columbus, district and 
charter results were comparable. In the other five cities, district schools outperformed charters. 
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• A significantly higher percentage of urban district schools received the state's highest ratings (Effective, 
Excellent, and Excellent with Distinction): 43 percent compared with 20 percent of charter schools. And a 
higher percentage of charters received the state's lowest ratings (Academic Watch and Academic 
Emergency): 49 percent of charter schools, compared with 34 percent of district schools. 

• Urban charter schools showed stronger value-added growth than their district counterparts, with 79 
percent of charters making expected or above expected growth in reading, compared with 68 percent of 
district schools. 

• Among the state's growing number of e-schools, smaller e-schools (those serving fewer than 500 
students) made significant gains in academic achievement in reading and math, largely closing the gap 
between their performance and that of larger e-schools. 
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Introduction 



This report compares the 2009-10 performance of Ohio's urban charter schools with that of comparable district 
schools. Comparisons are made in four sections: 

• Overall achievement and growth 

• Performance trends over time 

• Ratings on state accountability systems 

• Performance and growth across districts and statewide 

For 2009-10, we track the results of two types of charter schools in the Buckeye State. First are "e-schools" or 
"virtual schools," meaning they provide instruction to students online at home. These schools serve students from 
districts across the state. The other type of charters are "brick and mortar" schools located primarily in Ohio's 
eight major urban districts (the Ohio 8); Akron, Canton, Cincinnati, Cleveland, Columbus, Dayton, Toledo, and 
Youngstown. By law, these schools draw their students almost entirely from the large, urban school districts in 
which they are physically located. 

To provide a fair comparison, this report compares e-school performance to that of non-charter public schools 
statewide, and urban charter school performance to that of the urban school districts in which these schools are 
located. This approach provides us with an "apples-to-apples" comparison of student achievement based on 
comparable student pools from which the charter schools can draw. 

Overall Achievement and Growth 

Urban Elementary and Middle Schools 

Ohio's reporting system makes it possible to examine elementary and middle school performance on two 
dimensions: achievement and growth. Ideally, schools will have high proportions of their students achieving at 
grade level and their students will be making measurable growth or progress in test scores over the course of the 
school year. 

Ohio summarizes school achievement using a "Performance Index." This score averages a school's student 
achievement in all tested subjects in grades 3-8, with the most weight given to students who exceed state 
standards. The Performance Index runs on a scale from 0 to 120, with a state goal of 100 for all schools. 

For the third year in a row, Ohio also rated each school's "value added": a measure of how much growth students 
made in both reading and math over the course of one year, compared to how much the state would expect them 
to gain. Using this information, Ohio determines if each school made Above Expected Growth, Expected Growth or 
Below Expected Growth. Value-added scores are only available for some elementary and middle school grades in 
Ohio. 1 

Chart 1 compares the performance of Ohio's urban charter schools on both of these dimensions with that of 
district schools in the eight major urban school districts. The upper-right section of the matrix is the ideal: high 
achievement and high growth. Each square represents an urban charter or district elementary or middle school 
(high schools do not receive a value-added score in Ohio). The vertical placement of each square represents a 



1 This year Ohio made two changes to its value added methodology. For more information on these changes see: 
http://www.edexcellence. net/gadfly/index. cfm?issue=591#a6341. 
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school's achievement; the higher a square, the higher the achievement. The horizontal location of each square 
represents a school's value-added category only (that is, a square on the left side of a box does not necessarily 
have lower value-added than one on the right; they are both in the same value added category). For this report, 
the bottom performance tier includes schools with Performance Indexes below 80, the middle tier includes schools 
with Performance Indexes between 80 and 99.9, and the upper tier includes schools with Performance Indexes of 
100 or more. Blue squares represent Ohio's urban charter schools and red squares represent Ohio 8 district 
schools. For an analysis of how these results varied by district, see Appendix 1. 



Chart 1: Urban Charter Schools vs. Ohio 8 District Schools, Performance Index and Growth in 
Reading and Math, 2009-10 
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As Chart 1 shows far too many urban schools, both charter and non-charter, failed to reach the highest levels of 
performance in 2009. Only three charters (two percent) and six district schools (two percent) had high growth and 
high achievement. Meanwhile, 20 percent of charter and 21 percent of district schools fell in the bottom-left 
corner of the graph, indicating both low growth and low achievement, an increase of six and nine percentage 
points, respectively, over last year. 

In general, performance was similar in charter and district schools. Both types of schools performed better on 
measures of growth than on measures of achievement. Forty-four percent of charter schools and 30 percent of 
district schools exhibited above expected growth in 2009. In contrast, only seven percent of charter schools and 
four percent of district schools exhibited high achievement in 2009. Meanwhile, 62 percent of charter schools and 
59 percent of district schools fell into the lowest tier of the Performance Index, while 26 percent of charters and 37 
percent of district schools exhibited below-expected growth. 

Table 1, below, shows the names and locations of the highest performing charter and district schools in Ohio's 
eight major urban districts. In 2009, these schools had both high growth and high achievement. 
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Table 1: High-Performing Schools in Ohio 8 Urban Districts, 2009-10 





Schools with High Growth and Achievement 


Key: 


Akron 


Columbus 


Charter Schools 
District Schools 



Miller-South Visual Performing 
Arts Middle 


Clinton Elementary School 


Canton 


Dayton 


Portage Collab Montessori 


Cincinnati 


Toledo 


Covedale Elementary 
Walnut Hills High 


Toledo School for the Arts 


Cleveland 


Youngstown 


Marcus Garvey Academy 



Old Brooklyn Community School 
Whitney Young School 



One important question to ask is what happens to low-performing schools over time? In our 2007-08 report, we 
noted that over a third of urban charter and district schools were located in the bottom-left section of the matrix; 
that is, they had below expected growth and low performance. Chart 2 examines where these low-performers 
were located in 2009-10, two years later. It examines only charter and district schools that had below expected 
growth and low performance in 2007-08. 

Chart 2: 2009-10 Performance Index and Growth Levels in Reading and Math for 2007-08's Low-Performing, 
Low-Growth Schools 
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Note: Each blue and red box on this chart represents a school that fell into the bottom-left section of this chart 
(low-growth, low-achievement) in 2007-08. The boxes are placed to show each school's growth and achievement in 
2009-10. 



Compared with the lowest performing district schools, the lowest performing charters showed both more growth 
and greater achievement. Eight (23 percent) of the lowest performing charter schools in 2007-08 made above 
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expected growth and reached moderate achievement in 2009-10. Among district schools, only three (two percent) 
are now making above expected growth and moderate achievement. Schools that moved out of the bottom-left 
cell seemed to do so by showing more growth - 69 percent of the lowest performing charter schools and 67 
percent of the lowest performing district schools made expected to above expected growth this year. Still, despite 
some progress, far too many low-performing schools continue to languish. Seventy-one percent of charter schools 
and 90 percent of district schools remained in the bottom Performance Index tier, while 29 percent of charters and 
30 percent district schools remained in the bottom Performance Index tier and achieved below expected growth in 
2009-10. 

Elementary and Middle Charter E-Schools vs. Schools Statewide 

Chart 3 shows a comparison of the state's charter e-schools and non-charter public schools statewide on 
achievement and growth. This chart only displays information about the state's nine e-schools serving middle and 
elementary students. High schools in Ohio do not receive a value-added classification. Including high schools, 
Ohio has 27 charter e-schools serving 29,094 students (a full 31 percent of the state's charter population). In Chart 
3, each square represents an individual school, either one of the state's eight large charter e-schools (those with 
enrollment of 500 or more), one of the state's small e-schools (with enrollment lower than 500), or a traditional 
district school statewide. 

Chart 3: E-School vs. Statewide Performance Index and Growth in Reading and Math, 2009-10 
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Chart 3 shows that the only small e-school with value-added data made above expected growth, although it also 
scored in the lowest Performance Index tier (below 80). Among the eight large e-schools, just two made above- 
expected growth, while three made expected growth and the remaining three made below expected growth. No 
charter e-schools performed in the top tier of the Performance Index compared to 31 percent of non-charter 
public schools statewide. 
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Urban Charter School vs. Ohio District School Performance Over Time 



Looking at long-term trends can help illuminate the trajectory of student achievement in Ohio's charter and urban 
district schools. Charts 4 and 5 examine how the performance of students in Ohio 8 charter and district schools 
has changed over time. This comparison uses weighted averages that take into account the percentage of charter 
students in each grade and city when comparing their performance to that of district schools. For a more detailed 
explanation, see Appendix 2. 

When student achievement data were first made available by the Ohio Department of Education in 2001-02 urban 
charter school performance lagged significantly behind that of the surrounding district schools. Between 2001-02 
and 2005-06, charter school performance in both reading and math rose to the point where it was very similar to 
that of the urban systems where the charter schools reside. Since then, urban charter performance has leveled off 
and largely mirrored performance in nearby district schools. This year, however, charter schools performed 
marginally better than their district counterparts in reading and math, outscoring districts by three percentage 
points in reading and one percentage point in math. 

Analyzing only statewide performance in reading and math does little to reflect the significant variation that exists 
among charter and district schools within different cities. Later in this report, Charts 10 - 13 provide city-level 
performance data for further comparison among cities in reading, math, science, and writing. 



(This space left intentionally blank) 
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Chart 4: Urban Charter School vs. Ohio 8 District School Performance over Time in Reading 
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Chart 5: Urban Charter School vs. Ohio 8 District School Performance over Time in Math 
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Urban Charter Schools vs. Ohio 8 District Schools, Performance On State 
Accountability Systems 



Another way to compare performance is to examine how schools fared via state accountability ratings. Ohio's 
accountability system places schools into one of six categories based on a range of performance measures. From 
highest to lowest, those performance designations are: Excellent with Distinction, Excellent, Effective, Continuous 
Improvement, Academic Watch, and Academic Emergency. Charts 6 and 7 show the percentages of Ohio 8 charter 
and district schools, respectively, that fell into each state category in 2009-10. 

Across urban charter schools a higher percentage of schools received more favorable designations, compared to 
last year. The number of schools labeled Academic Emergency dropped by seven percentage points, while the 
number of schools deemed Excellent improved by two percentage points. Still, nearly half of all urban charters are 
designated as Academic Emergency or Academic Watch. In contrast, 34 percent of Ohio 8 district schools fell in 
those categories in 2009-10. Like their charter counterparts, fewer Ohio 8 district schools were designated 
Academic Emergency compared to last year, while more schools were deemed Excellent or Excellent with 
Distinction. 

Chart 6: Percent of Ohio 8 Charter Schools Chart 7: Percent of Ohio 8 District Schools 
by Performance Designation, 2009-10 by Performance Designation, 2009-10 
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Charts 8 and 9 show the percentage of e-schools and non-charter schools statewide with each performance 
designation. E-schools followed the same trend as urban charters and Ohio 8 schools, with more schools receiving 
higher designations and fewer receiving the lowest designations. More than half of the state's e-schools received 
a rating of Continuous Improvement, compared with 14 percent of schools statewide. Few e-schools received the 
state's highest ratings, with only 13 percent rated Excellent or Effective compared with 77 percent of non-charter 
district schools statewide. 
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Chart 8: Percent of E-Schools by Performance Chart 9: Percent of State Schools by Performance 
Designation, 2009-10 Designation, 2009-10 
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SUBJECT-BY-SUBJECT CHARTER SCHOOL PERFORMANCE AND GROWTH 

In the following sections, we compare the performance of charter schools on statewide tests in reading, math, 
science, and writing. The first section compares performance of urban charter schools to that of Ohio 8 district 
schools. The second section compares performance of e-schools to that of non-charter public schools statewide. 

Urban Charter Schools vs. Ohio 8 District Schools 

Charts 10 - 13 compare the average performance of charter schools located in the Ohio 8 cities to the average 
performance of their district counterparts on state tests in reading, math, science, and writing. Like the change 
over time data in Charts 4 and 5, these comparisons use weighted averages that take into account the percentage 
of charter students in each grade and district. 

In the eight cities as a group, charter schools slightly out-performed district schools in every subject but writing, 
driven largely by strong results among charter schools in Cleveland and Dayton. For both school types, 
performance was strongest in writing, with 69 and 74 percent of students proficient or advanced in charter and 
district schools, respectively. Performance was weakest in science, with 39 and 40 percent of students at least 
proficient in district and charter schools, respectively. 

In two cities, Cleveland and Dayton, charter schools outperformed district schools by margins ranging from nine 
percentage points in reading to 15 percentage points in science. In Columbus, charter and district performance 
were roughly similar, with no difference or one percentage point difference in each subject. In the other five 
urban districts, charter performance lagged behind district schools. Charter performance was weakest in 
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% of Studenta Proficient or Advanced 



Youngstown, where charters performed as abysmally as district schools in reading and math, but were 
outperformed by their district peers by 34 percentage points in writing and seven percentage points in science. 

Chart 10: Urban Charter School Performance vs. Ohio 8 District Performance in Reading, 
2009-10 
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Chart 11: Urban Charter School Performance vs. Ohio 8 District Performance in Math, 
2009-10 
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Chart 12: Urban Charter School Performance vs. Ohio 8 District Performance in Science 
2009-10 
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Chart 13: Urban Charter School Performance vs. Ohio 8 District Performance in Writing, 
2009-10 
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While achievement scores measure the extent to which students are proficient (meeting state standards), value- 
added scores measure how much students have learned (their growth) over the course of the year. Ohio uses a 
value-added system to determine whether students have made above expected growth, expected growth, or 
below expected growth in reading and math. Charts 14 and 15 compare the percentage of charter and district 
schools making expected or above expected growth in reading and math. 

In reading, more urban charter schools made expected or above expected growth (79 percent) than Ohio 8 
schools (68 percent). In only two cities - Cleveland and Youngstown - did a notably higher percentage of Ohio 8 
schools meet or exceed expected growth compared to charter schools. 

Across Ohio 8 districts, more urban charters also made expected or above expected growth in math - 74 percent 
of urban charter schools compared to 63 percent of Ohio 8 district schools. The opposite held true in three cities - 
Akron, Cincinnati, and Youngstown. 



Chart 14: Urban Charter Schools Making Expected or Above Expected Growth vs. Ohio 8 
District Schools Making Expected or Above Expected Growth in Reading, 2009-10 
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Chart 15: Charter Schools Making Expected or Above Expected Growth vs. Ohio 8 District 
Schools Making Expected or Above Expected Growth in Math, 2009-10 
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Note: Only two charter schools in Canton received a value-added rating in 2009-10. 



E-Schools vs. Non-Charter Public Schools Statewide 

Chart 16 compares the performance of students in charter e-schools to the performance of students statewide. 
Unlike "brick and mortar" charter schools, e-schools cannot be compared to the districts where they are located 
because they can enroll students throughout the state. Among the 27 charter e-schools currently operating in 
Ohio, only 8 serve more than 500 students. These are represented in Chart 16 by the green bars. Blue bars 
represent smaller e-schools, with red representing non-charter schools statewide. 

In 2009-10, district schools statewide outperformed e-schools in every subject, but the gap between small e- 
schools and schools statewide shrank in every subject. Small e-schools showed the largest improvements in 
reading, where 74 percent of students were proficient in 2009-10, compared with only 53 percent in 2008-09. 
Large e-schools outperformed their smaller counterparts in reading, science, and writing; however, small e-schools 
outperformed large e-schools in math by five percentage points. 



Ohio Urban School Performance Report, 2009-10 



Prepared by Public Impact 



15 



Chart 16: E-school Performance vs. Statewide Performance, 2009-10 




■ Large E-school ■ Small E-school ■ Statewide 



Chart 17 shows the percentage of small e-schools, large e-schools, and district schools statewide that made 
expected or above expected growth in 2009-10. In reading, five of the eight large e-schools (63 percent) made or 
exceeded expected growth in reading while 67 percent of district schools statewide made or exceeded expected 
growth. More large e-schools made or exceeded expected growth in math compared to district schools statewide, 
with 75 percent of large e-schools making expected or above expected growth, compared with 64 percent of 
district schools statewide. Value-added data are available for just one small e-school (not shown), which exceeded 
expected growth in both math and reading in 2009-10. 



Chart 17: Percentage of E-schools vs. Non-Charter Schools Statewide Making Expected or 
Above Expected Growth in Reading and Math, 2009-10 
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Note: Small e-schools are not shown in this chart because only one small e-school, Goal Digital Academy, had 
value-added data in 2009-10. Goal Digital Academy made expected growth in both math and reading. 



Ohio Urban School Performance Report, 2009-10 



Prepared by Public Impact 



16 



Appendix 1: District-by-District Performance and Growth 



In the following section, we examine the performance of urban charter schools and Ohio 8 district schools within 
the same city on two dimensions: performance and growth. The upper-right section of the chart is the ideal: high 
achievement and high growth. Each square represents an urban charter or district elementary or middle school in 
the city (high schools do not receive a value-added score in Ohio). Blue squares represent charter schools and red 
squares represent district schools. 



Chart Al: Performance Index and Growth in Reading and Math in Akron, 2009-10 
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Chart A2: Performance Index and Growth in Reading and Math in Canton, 2009-10 
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Chart A3: Performance Index and Growth in Reading and Math in Cincinnati, 2009-10 
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art A4: Performance Index and Growth in Reading and Math in Cleveland, 2009-10 
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Chart A5: Performance Index and Growth in Reading and Math in Columbus, 2009-10 
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Chart A6: Performance Index and Growth in Reading and Math in Dayton, 2009-10 
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Chart A7: Performance Index and Growth in Reading and Math in Toledo, 2009-10 

Below Expected Growth Expected Growth Above Expected Growth 



120 



100 



X 

CD 

"D 

c 



CD 

Q_ 



80 



60 - 



40 



□ 



-b-b- 



d 3 



-Q-b- 



& 

□ □ □ 
□ 



XL 



% □ 



CD □ 

□ 



0 



□ 



□ 



□ 

□ 






□ 



-Q- 



□ 



□ 



IT 

□ □ 



□ 

□ □ 



W 



□ 



□ 



□ Charter 
Schools 



□ District 
Schools 



Chart A8: Performance Index and Growth in Reading and Math in Youngstown, 2009-10 

Below Expected Growth Expected Growth Above Expected Growth 



120 



100 



X 

CD 

"O 

c 



c 

03 

E 



O) 

CL 



80 



60 



40 



□ 



□ 



□ 



□ 



iF 



□ 



□ 



□ 



□ 



□ Charter 
Schools 



□ District 
Schools 



Ohio Urban School Performance Report, 2009-10 



Prepared by Public Impact 



20 



Appendix 2: Methodology 

Charts 1-3: Performance Index and Growth in Reading and Math, 2009-10 

Charts 1-3 display how schools were distributed based on their Performance Index scores and value-added 
classifications. Schools located in the top-right section have Performance Index scores above 99.9, and a value- 
added classification of Above Expected Growth. Schools in the middle-center section have Performance Index 
scores between 80 and 99.9 and a value-added classification of Expected Growth and schools in the bottom-left 
section have Performance Index scores below 80 and a value-added classification of Below Expected Growth. 

The vertical location of each square is determined by the Performance Index score of each school. The horizontal 
location of each square is determined by each school's value-added category. For instance, if a school received a 
value-added designation of Above Expected Growth it would be located in the right column. To help differentiate 
among squares, random variance was introduced into horizontal coordinates to create space between squares. 
While the placement of squares into columns is relevant, the horizontal location of squares within sections is 
irrelevant. That is, a square on the left side of a box does not necessarily have a lower value added than one on 
the right; they are both in the same value-added category. 

The segmentation of Performance Index scores into three categories is based on Ohio's school and district ratings 
system. Under Ohio's school and district ratings system, schools or districts with a Performance Index score above 
99.9 are able to receive a designation of Excellent. Schools or districts with a Performance Index score between 80 
and 99.9 can receive a rating of Effective or Continuous Improvement, and schools or districts with Performance 
Index scores below 80 can receive a rating of Academic Watch or Academic Emergency. 

Charts 4 & 5: Urban Charter School Performance over Time in Reading and Math 
Charts 4 and 5 use weighted averages to compare the performance of urban charter schools to the performance of 
their surrounding district schools. For example, if in 2009-10 30 percent of charter students were in third grade, 
then third graders in district schools would be counted as 30 percent of the district average. Similarly, if 30 
percent of charter students were in Akron, then Akron city district students would be counted as 30 percent of the 
district average as well. 

Charts 10 - 13 & 16: Charter School Performance vs. Non-Charter Performance, 2009-10 
These charts compare the performance of charter schools to the performance of their comparable non-charter 
district schools. Each analysis uses weighted averages that take into account the percent of charter students in 
each grade and city when comparing their performance to that of district schools. For example, if 30 percent of 
the charter students in Dayton were in third grade, third graders in the Dayton City School District would be 
counted as 30 percent of the district average as well. 

Charts 14 - 15 & 17: Charter School Growth vs. Non-Charter Growth, 2009-10 
Ohio measures growth using a value-added assessment: a measure of how much progress students made in 
reading and math over the course of one year compared to how much the state would expect them to gain. It is 
possible for schools to be classified as making Above Expected Growth , Expected Growth , or Below Expected 
Growth. Charts 14, 15 and 17 show the percentage of charter schools and comparable district schools that made 
Above Expected Growth and Expected Growth. 
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